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FOREWORD 


This  document  contains  the  results  of  an  early  assessment  of 
the  Location  Analysis  Applications,  a  module  of  the  AirLand 
Battle  Management  (ALBM)  Advanced  Technology  Demonstration  (ATD) 
prototype,  version  1.2.  ALBM  ATD  is  a  program  to  develop  deci¬ 
sion  aid  prototypes  to  support  Army  division-level  tactical 
planning.  This  assessment  is  one  of  a  series  of  life  cycle 
assessments  of  ALBM  ATD  being  conducted  by  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ART)  during  the 
development  of  the  system.  The  results  will  be  used  by  the 
developer  and  government  sponsors  of  ALBM  ATD  to  guide  further 
development  of  the  system. 

The  research  was  conducted  under  the  ARI  research  task 
entitled  "Support  for  Command  Control  Research."  The  assessment 
was  in  support  of  the  Combined  Arms  Command  (CAC) ,  the  program's 
user  representative.  A  Memorandum  of  Agreement  was  in  effect 
with  the  Combined  Arms  Combat  Developments  Activity,  "Development 
and  Implementation  of  the  Future  Battle  Laboratory,"  dated 
30  June  1989.  The  results  of  this  review  were  briefed  to  per¬ 
sonnel  from  the  Battle  Command  Battle  Laboratory,  Combined  Arm.* 
Command;  Communications  and  Electronics  Command;  Lockheed;  and 
MITRE  on  7  January  1993.  Brigadier  General  Anderson,  Deputy 
Commanding  General  for  Combat  Developments,  Combined  Arms  Center, 
was  briefed  on  the  findings  presented  in  this  report  on 
25  January  1993. 


EDGAR  M.  JOHNSON 
Director 
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EVALUATION  OF  THE  AIRLAND  BATTLE  MANAGEMENT  ADVANCED  TECHNOLOGY 
DEMONSTRATION  PROTOTYPE  VERSION  1.2:  ASSESSMENT  OF  THE 
LOCATION  ANALYSIS  APPLICATIONS 


Summary 

In  this  assessment,  subject  matter  experts  (SMEs)  judged  the 
usability  of  the  Location  Analysis  Applications  (LAAs) ,  a 
component  of  the  AirLand  Battle  Management  Advanced  Technology 
Demonstration  (ALBM  ATD) .  The  study  was  performed  as  a  part  of 
the  Army  Research  Institute's  (ARI)  support  of  the  Battle  Command 
Battle  Laboratory. 

Six  SMEs  participated  in  individual  assessment  sessions. 
Following  a  description  of  the  LAAs  by  the  evaluator,  SMEs 
completed  a  written  questionnaire.  In  addition,  SMEs  were 
encouraged  to  ask  questions  and  make  verbal  comments  that  were 
recorded  and  used  in  subsequent  data  analyses. 

Nineteen  LAAs  were  evaluated  for  their  operational 
usability.  The  results  from  this  study  showed  that 

•  four  applications  were  rated  highly  or  rather  usable, 

•  four  applications  were  rated  rather  usable  to  borderline 
usable, 

•  seven  applications  were  rated  as  having  borderline 
usability, 

•  four  applications  were  rated  as  unusable. 

SME  suggestions  for  improvement  stressed  consistent 
presentation  of  legends  with  clear  definitions  of  displayed  data, 
clearer  delineation  of  multiple  features  being  displayed,  and 
increased  selectivity  of  features  to  be  displayed.  The  SMEs  felt 
that  if  these  suggestions  were  implemented,  the  overall  usability 
of  the  applications  would  increase  considerably. 

It  is  concluded  that  4  of  the  19  LAAs  are  usable  in  their 
current  form,  although  improvements  would  be  worthwhile.  An 
additional  11  applications  would  be  usable  if  suggested 
improvements  were  made  to  the  LAA  functions.  It  is  recommended 
that  LAA  functions  be  improved  before  being  provided  to  units  for 
operational  use. 
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Introduction 


Overview 

This  report  documents  one  of  six  assessments  of  version  1.2 
of  the  Airland  Battle  Management  (ALBM)  Advanced  Technology 
Demonstration  (ATD)  prototype  conducted  by  the  U.S.  Army  Research 
Institute  (ARI) .  The  assessments  are  part  of  a  set  of  life  cycle 
evaluations  being  conducted  on  the  ALBM  ATD  prototype  as  it  is 
being  developed.  The  purpose  of  the  life  cycle  evaluations  is  to 
provide  user  and  subject  matter  expert  feedback  to  the  government 
sponsor  and  contractor  developer  in  order  to  guide  the  design  and 
development  of  the  system  and  to  provide  information  for 
management  decisions.  In  this  way,  it  is  hoped  that  the  final 
operational  system  will  have  capedsilities  that  will  improve  user 
performance. 

The  six  assessments  conducted  on  the  version  1.2  prototype 
include  knowledge  base  reviews  of  four  applications,  a  human 
factors  assessment  of  the  interface,  and  a  user  and  SME  review  of 
demonstrated  prototype  capabilities.  In  addition  to  this  report, 
these  assessments  are  documented  in  separate  ARI  reports 
(Flanagan,  1993;  McKeown,  1993a;  Riedel,  McKeown,  Flanagan,  & 
Adelman;  Rappold,  &  Flanagan  1993;  Riedel,  Flanagan,  Van  Hemel, 

&  Rappold,  1993) .  This  report  contains  the  results  of  an  early 
review  of  the  Location  Analysis  Applications,  one  module  of  the 
ALBM  ATD  prototype. 

Description  of  ALBM  ATD 

ALBM  ATD  is  a  Training  and  Doctrine  Command  (TRADOC)  and 
Army  Materiel  Command  (AMC)  program.  Its  purpose  is  to  develop 
decision  aid  prototypes  based  on  advanced  technologies  and 
transition  them  to  the  Army  Tactical  Command  and  Control  System 
(ATCCS) .  The  decision  aids  are  intended  to  support  corps, 
division  and  brigade  level  commanders  and  their  staffs  in 
tactical  planning  operations.  The  Communications  and  Electronics 
Command  at  Fort  Monmouth  is  responsible  for  the  overall 
management  of  the  program;  the  Combined  Arms  Command,  Combat 
Developments  at  Fort  Leavenworth  is  the  users'  representative 
responsible  for  functional  requirements,  knowledge  elicitation 
with  subject  matter  experts,  and  the  operational  evaluations; 
and  the  Program  Executive  Office  for  Command  and  Control  Systems 
(PEO-CCS)  is  responsible  for  integration  with  the  Army  Tactical 
Command  and  Control  System  (ATCCS) . 

Two  Force  Level  Control  (FLC)  Advisors  are  currently  under 
development  as  part  of  the  ALBM  ATD  system  -  MET4  and  FITE.  MET4 
(Mission,  Enemy,  Terrain,  Troops  and  Time  Available  Applications) 
is  intended  to  aid  commanders  and  their  staffs  from  brigade 
through  corps  to  analyze  the  area  of  operations  and  to  assess  the 
enemy  and  friendly  capabilities.  FITE  (Force  Interactive 
Tactical  Evaluator)  interacts  with  MET4  to  aid  commanders  and 
their  staffs  to  develop,  wargame,  and  compare  COAs. 
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MET4  has  four  basic  components. 

-  Battlefield  Area  (BA)  Component  assists  commanders  and 
staff  to  analyze  the  terrain  and  develop  and  analyze  avenues  of 
approach . 

-  The  Enemy  and  Situation  Capabilities  (ESC)  Component 
interacts  with  the  other  MET4  components  to  aid  commanders  and 
staffs  to  anticipate  enemy  operations.  Its  principal  focus  is  on 
probable  enemy  courses  of  action. 

-  Friendly  and  Situation  Capabilities  (FSC)  Component 
interacts  with  other  MET4  components  to  assist  commanders  and 
staffs  to  analyze  missions  received  from  higher  headquarters  to 
assess  the  friendly  situation  and  to  determine  the  general 
ability  of  the  unit  to  accomplish  its  assigned  mission.  The 
focus  is  on  projecting  friendly  unit  readiness  and  capabilities. 

-  The  Execution  Monitor  (EM)  Component  interacts  with  ATCCS 
components,  FITE,  other  MET4  components,  and  other  decision  aids 
to  aid  commanders  and  staffs  to  monitor  current  operations.  It 
alerts  commanders  and  staffs  when  the  current  operation  deviates 
from  the  Operations  Order  (OPORD) ,  aids  commanders  and  staffs  to 
determine  when  orders  should  be  issued  to  implement  new  phases  or 
branches  provided  for  in  the  current  plan,  and  when  modifications 
of  the  current  plan  or  replanning  are  necessary. 

The  second  ALBM  ATD  module,  FITE,  interacts  with  MET4  to  aid 
commanders  and  staffs  to  develop,  wargame,  and  compare  Courses  of 
Action  (COAs) .  It  also  aids  commanders  and  staffs  to  properly 
synchronize  operations  of  subordinate  and  supporting  units  in 
order  to  concentrate  combat  power  at  the  critical  place  and  time 
to  accomplish  the  commanders  intent.  Its  principal  focus  is  on 
COA  development,  preparation  of  the  COA  sketch,  COA  analysis 
(wargaming)  and  comparison,  and  on  an  execution  synchronization 
matrix. 

Description  of  Location  Analysis  Applications 

The  set  of  LAAs  a  component  of  MET4.  It  contains  the 
following  terrain  analysis  applications: 

Terrain  Overlays  for 
Elevation  Bands 
Vegetation 
Drainage 
Obstacles 

Surface  Configuration 
Surface  Materials 
Transportation 

Cross  Country  Mobility 

Cover  (26-50%,  51-75%,  76-100%) 

Mobility  Arrows 
Query  for  Line-of-sight 


4 


Query  for  Distance  Along  a  Path 

Query  for  Elevation  at  a  Point 

Location  Analysis  (i.e  Tactical  Assembly  Area) 

Query  for  Terrain  Attributes  (points  and  areas) 
Query  for  Built-up  Area 
Query  for  Slopes  >  10% 

Query  for  Dense/Old  Forest 
Query  for  NO-GO  Gaps 

Appendix  B  contains  screen  images  of  these  applications. 

Assessment  ' 


It  is  beyond  the  scope  of  this  study  to  evaluate  the 
completeness  and  accuracy  of  the  data  bases  which  provide  the  19 
LAAs  (i.e.,  the  Engineering  Topographic  Laboratory  (ETL)  and  the 
Defense  Mapping  Agency  (DMA)  data  bases) .  Also,  there  are  no 
algorithms  for  processing  the  data;  rather,  the  data  are 
extracted  from  the  data  bases  for  graphic  presentation  to  the 
user.  Therefore,  a  single  key  issue  was  used  to  evaluate  the 
nineteen  LAAs: 

Is  the  module  usable,  in  terms  of  information  content  and 
format? 

This  study  involved  obtaining,  recording,  and  analyzing 
feedback  from  SMEs  on  the  current  usability  of  LAAs.  The 
procedure  for  conducting  the  study  centered  on  obtaining 
reactions  to  the  existing  application,  rather  than  on  obtaining 
suggestions  on  how  LAAs  should  function.  There  was  no  attempt  to 
elicit  design  suggestions,  although  some  were  volunteered  and 
recorded  during  the  study.  As  a  result,  this  study  present 
problems,  deficiencies,  and  omissions  in  the  functionality  of 
LAAs,  without  always  presenting  approaches  for  overcoming  these 
problems.  It  is  hoped  that  system  engineers  and  system  designers 
will  be  able  to  review  the  results  contained  herein,  and  develop 
appropriate  enhancements  or  corrections  to  the  current  design  of 
the  LAAs. 
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Met-hod 


Subject  Matter  Experts 

Six  subject  matter  experts  participated  in  the  assessment. 
They  had  the  following  qualifications: 

•  Rank  of  Major  or  Lieutenant  Colonel. 

•  Worked  regularly  in  a  division  G2  section. 

.  Performed  Intelligence  Preparation  of  the  Battlefield 
and  Terrain  Analysis  functions  at  division/corps  level. 

.  Graduated  from  the  Command  and  General  Staff  Officer 
Course  (CGSOC) . 

•  Possessed  a  working  familiarity  with  FM  5-33,  "Terrain 
Analysis",  July  1990?  FM  34-130,  "Intelligence 
Preparation  of  the  Battlefield",  May  1989;  and  ST  100- 
9,  "The  Command  Estimate  Process",  Chapter  7, 
"Intelligence  Preparation  of  the  Battlefield",  July 
1992. 

Appendix  D  describes  the  individual  SMEs  in  more  detail. 
Documentation  of  LAA  Procedures  and  Products 

Descriptions  of  the  LAA  procedures  and  products  were 
obtained  from  Software  User's  Manual  for  the  ALBM  ATD  Force  Level 
Control  Advisor  System  (Lockheed,  30  May  1992) ,  examination  of 
LAA  displays  and  features,  and  personal  communication  with 
developer  personnel. 

Materials 

Briefing  materials  and  questionnaires  were  developed  as  part 
of  this  study.  The  briefing  materials  provided  a  uniform, 
repeatable  presentation  to  the  SMEs  (see  Appendix  A).  They 
include  an  explanation  of  the  ALBM  ATD  program,  the  role  of  this 
study  within  the  ALBM  ATD  program,  and  an  overview  of  the 
nineteen  LAAs. 

To  illustrate  the  LAAs  process,  color  prints  of  the  ALBM  ATD 
workstation  screen  were  obtained  showing  the  various  LAAs  (see 
Appendix  B  for  black  and  white  versions  of  the  color  prints) . 
Color  prints  were  used  in  lieu  of  the  actual  workstation  to 
separate  the  soldier-machine  interface  function  and  system 
performance  aspects  of  the  ALBM  ATD  system  from  the  underlying 
usability  of  each  application.  In  this  way,  biases  regarding 
other  functional  aspects  of  the  ALBM  ATD  system  would  not 
influence  the  results  of  this  study. 

A  questionnaire  was  developed  based  on  the  key  issue 
identified  earlier.  The  questionnaire  required  ratings  of  each 
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application,  and  invited  written  continents.  Also,  a  questionnaire 
was  developed  to  obtain  demographic  information  on  each  SME,  and 
a  release  form  was  signed  by  each  SME  regarding  video-taping  of 
the  session  and  participation  in  the  study.  These  materials  are 
contained  in  Appendix  C. 

Procedure 

This  study  was  conducted  in  combination  with  a  knowledge 
base  assessment  of  a  related  ALBM  ATD  module,  AA  Generation  (see 
McKoewn,  1993) .  Each  participant  took  part  in  both  assessments. 
The  LAA  assessment  was  conducted  after  the  AA  Generation 
assessment  and  took  approximately  1  to  hours  to  complete.  All 
sessions  were  conducted  at  the  ARI  Field  Unit  Laboratory  at  Fort 
Leavenworth ,  Kansas . 

The  following  is  a  chronological  listing  of  activities 
performed  during  each  session. 

•  Description  of  the  laboratory  and  building  facilities. 

•  Release  form  (see  Appendix  C)  regarding  participation 
in  the  study  and  video-taping  of  the  session 
administered.  Demographic  questionnaire  administered. 

•  Presentation  of  briefing  (see  Appendix  A)  explaining 
the  study  purpose,  doctrinal  background,  and  overview 
of  LAAs.  Questions  and  comments  made  by  the  SME  are 
recorded  manually  by  the  researcher. 

•  Presentation  of  color  prints  of  each  application  (see 
Appendix  B) ,  which  illustrate  the  knowledge  base  of  AA 
Generation.  Questions  and  comments  from  the  SME  are 
recorded  manually  by  the  researcher. 

•  With  the  color  prints  of  the  application  available,  the 
SME  completes  the  Usability  Questionnaire.  Questions 
and  comments  from  the  SME  are  recorded  manually  by  the 
researcher. 

Data  Analysis 

Due  to  the  limited  number  of  SMEs  available  for  this  study, 
no  statistical  analyses  were  performed  on  the  data.  Instead, 
responses  were  analyzed  to  assess  the  SME's  subjective  appraisal 
of  the  LAAs  in  ALBM  ATD. 

Comments  concerning  the  LAAs*  function  as  a  whole  were 
treated  separately  in  the  analysis.  Verbal  comments  duplicating 
a  written  comment  by  the  same  SME  were  discarded.  Verbal  and 
written  comments  that  exactly  or  closely  replicated  comments  made 
by  another  SME  were  retained  and  categorized  to  indicate  a  degree 
of  consensus  on  a  topic. 


8 


Results 


The  data  were  compiled  and  categorized  to  facilitate 
analysis  and  clear  presentation  of  results.  The  results  consist 
of  usability  ratings  of  each  of  the  nineteen  LAAs,  comments  that 
apply  generally  to  all  of  the  LAAs,  and  comments  that  pertain  to 
individual  LAAs. 

Usability  Ratings 

Included  in  Appendix  E  are  SME  ratings  of  the  LAAs  plus 
written  and  verbal  comments.  Examination  of  this  data  shows  the 
degree  of  consensus  by  the  SMEs  on  the  usability  of  the  LAAs. 

For  fifteen  of  the  LAAs,  the  consensus  was  good.  Four  of  the 
LAAs  had  ratings  that  were  not  consistent.  These  four  LAAs  are: 

•  Slopes  over  10%, 

•  Oense/Old  Forest, 

•  Built-up  Areas,  and 

•  Terrain  Attributes  in  an  Area. 

To  summarize  the  relative  usability  of  the  LAAs,  a  mean 
rating  was  calculated  for  each  application.  These  mean  ratings 
are  shown  in  the  form  of  a  horizontal  bar  graph  in  Figure  1, 
ordered  from  the  most  usable  to  the  least  usable  LAAs.  The 
horizontal  lines  in  Figure  1  represent  LAAs  in  terms  of  perceived 
usability. 

Ai  can  be  seen,  applications  with  mean  ratings  of  rather  usable 
or  higher  were: 

•  Distance  Along  a  Path 

•  Elevation  Bands 

•  Elevation  at  a  Point 

•  Cross-Country  Mobility 

The  NO-60  Gaps  application  was  rated  the  lowest  with  a  mean 
score  less  than  Somewhat  Unusable. 

General  Suggestions 

This  section  provides  a  summary  of  data  collected  from  SMEs 
that  applies  to  a  broad  range  of  applications. 

Legend  with  Description.  Many  of  the  current  displays  have 
legends,  but  several  do  not.  All  SMEs  wanted  legends  on  all 
displays.  They  felt  that  many  of  the  current  legend  descriptions 
were  vague  or  missing.  Many  of  the  legend  displays  were  large, 
tending  to  obliterate  the  overlay. 
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Figure  1.  Mean  usefulness  ratings  of  Local  Analysis  Applications. 


Outlining  of  Polygons.  Some  of  the  displays  consist  of 
areas  filled  with  a  pattern,  without  a  perimeter  boundary.  Some 
of  the  colors  and  patterns  were  difficult  to  distinguish, 
particularly  when  viewed  over  the  background  EMAP.  The  SMEs  felt 
that  displaying  terrain  attributes  with  translucent  colors  and 
then  outlining  the  areas  would  be  preferable  to  the  current 
display. 

Selectivity  of  Display  Features.  Many  of  the  displays 
contain  multiple  features  (e.g.,  the  transportation  network  which 
contains  several  classes  of  roads,  the  vegetation  overlay  which 
includes  several  forms  of  vegetation) .  The  SMEs  wanted  the 
ability  to  select  which  of  those  features  to  include  in  the 
display.  For  example,  the  user  may  want  to  display  only  the 
high-speed  and  secondary  roads,  but  not  forest  trails. 

Similarly,  the  user  may  want  to  display  only  slopes  of  30%  -  45%, 
and  over  45%,  without  displaying  any  of  the  other  slope  bands. 

Ability  to  Easily  Distinguish  Between  Multiple  Features.  The 
SMEs  felt  that  many  of  the  displays  were  presented  in  a  fashion 
that  made  it  hard  to  discern  the  differences  between  multiple 
features  contained  in  one  display.  It  was  suggested  that  use  of 
translucent  colors  would  improve  the  ability  to  distinguish 
different  features.  If  translucent  colors  were  not  possible, 
then  solid  colors  with  the  ability  to  toggle  the  overlay  on  and 
off  would  be  acceptable.  Also,  the  ability  to  quickly  click  with 
the  mouse  on  a  feature  (i.e.,  without  going  through  menus)  and 
receive  a  short  description  of  the  selected  feature  was  suggested 
for  inclusion  in  the  system. 

Combining  of  Applications.  The  SMEs  suggested  that  certain 
overlays  should  be  logically  combined.  The  example  given  was  the 
Obstacle  overlay,  which  should  be  combined  with  the  Drainage  and 
Built  Up  Areas  overlays  since  they  all  are  tactically  related  in 
terms  of  restricting  movement.  Currently,  this  combining  can  be 
accomplished  by  requesting  simultaneous  display  of  the  three 
overlays.  The  SMEs,  however,  felt  that  the  tactical  similarity 
of  the  three  overlays  warrants  them  being  requested  with  a  single 
request,  as  long  as  the  capabilities  for  selectivity  and  the 
ability  to  distinguish  different  features  discussed  previously 
were  available. 

Use  of  Tactical  Terminology.  The  SMEs  generally  felt  that 
too  much  interpretation  was  necessary  in  using  the  applications. 
The  terminology  is  oriented  towards  that  data  base,  rather  than 
the  tactical  use  of  the  data.  For  example,  the  SMEs  thought  of 
Surface  Configuration  as  slopes,  and  thought  Dense/Old  Forest 
must  have  something  to  do  with  tree  diameter  and  spacing,  but 
they  weren't  sure. 

Suggestions  for  Specific  Applications 

Many  suggestions  were  made  that  were  specific  to  each 
application.  These  are  noted  below  in  the  order  of  perceived 
usability  shown  in  Figure  1.  (Note  that  the  general  suggestions 
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described  above  are  not  repeated  in  the  following  paragraphs.) 
Also  provided  below  is  the  rating  scale  given  for  each 
application  and  a  breakdown  of  how  it  was  rated  by  the  six  SMEs. 
Sample  screen  images  of  each  application  are  given  in  Appendix  B, 
and  are  referenced  as  a  figure.  For  purposes  of  reference  and 
orientation,  the  first  figure  in  Appendix  B,  Figure  B-1,  shows 
the  background  EMAP  without  any  of  the  LAAs  being  displayed. 

Distance  along  a  oath. 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly 

Unusable 

3 

3 

A  screen  image  was  not  used  to  explain  this  application; 
rather,  it  was  described  verbally  to  the  SMEs.  The  user  requests 
the  "Distance"  function,  clicks  with  the  mouse  to  create  a  multi¬ 
segment  line,  and  clicks  with  the  right  mouse  button  when 
complete.  At  that  time,  the  total  distance  of  all  segments  in 
the  line  is  shown  in  kilometers  at  the  top  of  the  display  window. 

Two  SMEs  suggested  that  the  user  should  have  the  ability  to 
hold  down  a  mouse  button  and  "drag"  the  cursor  along  a  curved 
path,  similar  to  some  PC  paint  programs.  The  distance  would 
readout  dynamically,  like  the  current  Lat-Long/UTM/Elevation 
readout.  One  of  the  SMEs  also  suggested  that  the  distance  be 
displayed  in  kilometers,  miles,  and  nautical  miles,  for 
interoperability  with  other  services  and  systems. 

Elevation  Bands  (See  Figure  B-2) . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly 

Unusable 

2 

4 

This  application  was  the  one  that  elicited  the  most  comments 
about  use  of  translucent  colors,  rather  than  solid  colors  that 
totally  cover  the  background  EMAP.  Adding  contour  lines  would 
greatly  improve  the  usability  of  the  display.  One  SME  also  asked 
that  the  high  points  or  hill  tops  be  denoted  graphically  within 
each  displayed  elevation  band. 


12 


Elevation  at  a  Point  (See  Ficmre  B-3^ . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly 

Unusable 

1 

4 

1 

One  SME  suggested  that  the  unit  of  measurement  ( i . e . , 
meters)  be  added  to  the  elevation  display.  There  were 
suggestions  for  two  additional  options  (1)  to  shorten  the  display 
to  just  the  grid  square  letters  and  either  6  or  8  digits  (e.g.  MC 
942  200,  versus  32U  MC  94219  19953)  and  (2)  to  display  a  marker 
and  the  elevation  value  at  the  cursor  position  such  that  a  series 
of  elevation  readouts  could  be  simultaneously  shown  on  the  screen 
(e.g. ,  o  312) . 

Cross  Country  Mobility  (See  Figure  B-4^ . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly 
Unusable  | 

6 

1 

Because  the  user  would  be  highly  unlikely  to  display  both 
the  wet  and  dry  CCM  overlays  simultaneously,  it  was  suggested 
that  a  separate  legend  be  supplied  for  each  separate  CCM  overlay. 
Also,  since  the  wet  CCM  overlay  contains  so  much  SLOW-GO  area,  it 
was  suggested  that  only  the  NO-GO  be  displayed,  or  that  the 
feature  selectivity  discussed  previously  be  available.  Another 
comment  dealt  with  the  limited  data  base  currently  available.  At 
other  times  of  the  year,  weather  conditions,  such  as  snow  and 
freezing  temperatures,  may  result  in  different  mobility  results 
than  is  currently  available. 

Surface  Configuration  fi.e.  Slopes;  See  Figure  B-5) . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly  | 
Unusable  | 

5 

1 

1 

The  SMEs  wanted  the  different  categories  of  slopes  to  be 
tactically  significant,  and  the  legend  to  reflect  that 
significance  as  well  as  the  percentage  of  slope.  For  example, 
show  slopes  that  are  not  passable  by  Ml's,  M113's,  and  2k  ton 
trucks . 
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Line  of  Sight  fSee  Figures  B-6  and  B-7) . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Border 1 ine 

Somewhat 

Unusable 

Decidedly 

Unusable 

1 

2 

3 

The  SMEs  felt  that,  in  many  instances,  a  Line-Of-Sight  (LOS) 
display  with  a  limited  arc  (i.e.,  in  a  specified  direction)  would 
be  adequate.  It  would  reduce  both  processing  time  and  display 
clutter.  Options  should  be  provided  to  consider  vegetation 
(i.e.,  on  a  seasonal  basis)  in  the  algorithm,  and  to  include 
electronic  LOS.  One  SME  felt  the  button  that  selected  display  of 
masked  or  unmasked  areas  was  confusing.  He  suggested  having  two 
radio  buttons  labeled  something  like  "visible”  and  "not  visible." 

Location  Analysis  fi.e..  Tactical  Assembly  Area; 

See  Figure  B-8) . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly 

Unusable 

3 

3 

There  were  a  variety  of  suggestions  for  this  application. 

It  was  generally  felt  that  too  much  data  was  presented  at  once, 
and  that  selectivity  of  features  displayed  was  necessary  as 
discussed  previously.  The  legend  should  be  oriented  towards  the 
graphic  display,  and  the  checklist  of  other  considerations 
currently  in  the  legend  should  be  separated  for  clarity,  possibly 
in  the  Operational  Help  display.  It  was  also  suggested  that  the 
application  should  scan  the  available  data,  and  display 
recommended  areas.  Another  suggestion  was  that  the  application 
should  be  restricted  to  a  defined  area  similar  to  the  query  for 
terrain  attributes  in  an  area,  rather  than  filling  the  entire 
window. 

Obstacles  fsee  Figure  B--91  . 


SME Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly 

Unusable 

4 

1 

1 

The  SMEs  felt  this  application  should  be  combined  with 
built-up  areas  and  drainage  (i.e.,  hydrology),  as  well  as 
including  tactical  obstacles.  This  would  present  a  more  complete 
picture  of  specific  restrictions  to  movement,  other  than  COM.  It 
was  suggested  that  standard  tactical  symbols  be  used  to 
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differentiate  the  types  of  obstacles,  and  that  amplifying 
information  on  a  specific  obstacle  be  available  as  a  query. 

Vegetation  fsee  Figure  B-10^ . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly  | 
Unusable  | 

3 

2 

1 

1 

It  was  suggested  that  there  are  too  many  categories 
presented,  and  the  differentiation  between  the  categories  was 
vague.  The  SMEs  also  suggested  that  the  categories  be  reduced 
and  the  display  organized  for  tactically  significance.  For 
example,  the  presentation  of  sparse  forest  and  dense  forest 
should  be  in  terms  of  relative  vehicle  mobility  in  those  areas. 

Drainage  (see  Figure  B-ll> . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly 

Unusable 

2 

4 

- 

SMEs  thought  that  the  standard  map  color  for  water,  "blue", 
should  be  used  rather  than  red.  They  also  wanted  more  hydrology 
information,  particularly  regarding  the  ability  to  ford  streams. 
Areas  that  are  unfordable  shoe: 1  be  highlighted  or  separately 
displayed.  One  SME  felt  the  application  was  not  usable  without 
accounting  for  varying  amounts  of  precipitation.  The  unfordable 
areas  are  restrictions  to  movement,  and  the  SMEs  felt  it  should 
be  combined  with  the  Obstacles  display. 

Cover  fsee  Figure  B-12) . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly  1 
Unusable  B 

2 

3 

1 

1 

The  terminology  for  this  display  is  not  appropriate.  The 
SMEs  recommended  calling  this  display  "Concealment,"  rather  than 
"Cover."  The  SMEs  suggested  that  options  be  added  to  account  for 
seasonal  variations  in  canopy  closure. 
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Transportation  (see  Ficnire  B~13i  . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly  | 
Unusable  \ 

1 

4 

1 

1 

SMEs  thought  this  display  could  be  improved  considerably  by 
providing  a  better  distinction  of  the  various  classes  of  roads. 
They  also  desired  the  addition  of  bridges  with  classification  and 
rail  nets. 

Terrain  Attributes  in  an  Area  ^see  Ficrure  B-14^  . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly  | 
Unusable  1 

2 

2 

1 

1  1 

SMEs  liked  the  general  idea  of  this  query,  but  wanted  it 
implemented  differently.  Only  data  desired  by  the  user  should  be 
displayed,  rather  than  all  data  in  the  area.  Also,  it  should  be 
presented  in  tactical  terms  (e.g.,  fording  sites,  bridges  and 
roads  by  classification,  slopes  by  mobility  class) . 

Mobility  Arrows  (see  Ficnire  B~15)  . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly  | 
Unusable  | 

2 

1 

3 

1 

Generally,  the  SMEs  thought  this  display  was  confusing  and 
difficult  to  read.  They  suggested  using  different  symbols  (e.g., 
parallel  lines  with  varying  gaps  between  the  lines  to  indicate 
the  mobility  by  different  force  echelons) .  One  SME  thought  the 
display  was  a  good  application  for  analyzing  the  validity  of 
generated  Avenues  of  Approach. 

Built-up  Areas  (see  Figure  B-16K 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

y 

Decidedly 

Unusable 

2 

2 

1 

1 
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SMEs  suggested  that  differentiation  of  density  of  built-up 
areas  be  provided,  and  the  key  for  those  densities  be  included  in 
the  legend. 

Surface  Materials  fsee  Figure  B-17^ . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly 

Unusable 

3 

3 

SMEs  thought  the  definitions  for  this  display  were  too  vague 
to  be  usable  (e.g.,  rough,  very  rough).  The  definitions  should 
be  expressed  in  terms  of  mobility  capability. 

Slopes  >  10%  ^see  Ficmre  B-18) . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

==== 

Decidedly 

Unusable 

1 

2 

2 

1  1 

SMEs  felt  this  query  was  redundant  with  Surface 
Configuration,  particularly  if  display  selectivity  is  provided. 
They  suggested  that  the  user  be  able  to  specify  the  slope 
threshold  percentage. 

Dense/Old  Forest  (see  Figure  B-19) . 


SME  Ratings 


Highly 

Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly 

Unusable 

1 

2 

1 

2 

The  definitions  for  this  display  were  too  vague  for  the 
SMEs.  They  also  felt  that  the  display  was  redundant  with  the 
EMAP  and  Vegetation  overlay. 

NO-GO  Gaps  (see  Figure  B-20K 


SME  Ratings 


1  Highly 

1  Usable 

Rather 

Usable 

Borderline 

Somewhat 

Unusable 

Decidedly 

Unusable 

1 

1 

1 

3 

This  display  appeared  to  be  identical  with  the  Drainage 
overlay,  and  was  redundant.  The  title  of  the  display  was 
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confusing,  since  the  SMEs  thought  the  title  referred  to  areas 
between  NO-GO  areas. 

Other  General  Comments 

During  the  course  of  the  assessment  sessions,  the  SMEs 
sometimes  made  comments  that  provided  additional  insight  into  the 
potential  usability  of  the  LAAs  in  the  field.  They  also  provided 
information  for  considering  possible  improvements  and 
enhancements  of  the  LAAs.  A  sximmary  of  general  comments  follows. 

Develop  Modified  Combined  Obstacles  Overlay  fMCOOK  In 
addition  to  the  selective  display  of  features  discussed  in  the 
previous  section,  one  SME  suggested  that  a  capability  be  provided 
to  edit  the  displays,  retain  the  results  of  queries,  and  add  text 
and  additional  tactical  symbols  (i.e.,  through  the  Overlay 
Editor) .  Then  the  displayed  terrain  data  could  be  combined  with 
control  measure  graphics  to  produce  an  MCOO. 

Make  Available  Relative  Aae  of  Maos  and  Terrain  Data.  Two 
SMEs  suggested  that  the  production  date  of  the  EMAP,  DMA  data, 
and  ETL  CCM  data  be  available  to  the  user.  The  SMEs  felt  there 
were  some  discrepancies  between  the  EMAP  and  some  of  the  DMA  data 
that  may  have  been  explained  by  the  passage  of  time. 


18 


Conclusions  and  Recommendations 

As  currently  implemented,  eight  of  the  LAAs  were  rated 
usable  by  the  SMEs,  and  eleven  were  rated  not  usable  or  of 
borderline  usability  (see  Table  1) . 


Table  1 

. 

bAAsi  rated  as  usable  and  not  usable 


Usable 

Not  Usable 

Distance  Along  a  Path 
Elevation  Bands 
Cross-Country  Mobility 
Surface  Configuration 
Line-of-Sight 

Location  Analysis 
Obstacles 

Vegetation 

Drainage 

Cover 

Transportation 

Terrain  Attributes  -  Area 
Mobility  Arrows 

Built-up  Areas 

Surface  Materials 

Slopes  >  10% 

Dense/Old  Forest 

NO-GO  Gaps 

The  SMEs  made  suggestions  that  apply  generally  to  each  of 
the  LAAs.  Implementation  of  these  suggestions  would  make  all  of 
the  applications  usable.  These  suggestions  are  as  follows: 

•  Legends  should  be  supplied,  when  missing,  and  the 
descriptions  made  more  precise. 

•  Cross-hatched  areas  should  be  outlined  to  provide 
clarified  feature  boundaries. 

•  Allow  the  user  to  be  able  to  select  which  features  to 
display. 

•  Provide  easy  discrimination  between  features  in  the 
displays. 

•  Some  applications  are  partially  redundant  and  should  be 
combined . 

•  Terms  in  menus  and  legends  should  reflect  tactical 
information,  not  data  names. 

Recommendations 

It  is  recommended  that  the  suggestions  listed  in  the 
previous  section  be  implemented.  If  the  ability  to  selectively 
display  features  is  implemented,  it  is  recommended  that  four  of 
the  LAAs  be  removed  from  ALBM  ATD,  since  their  features  are 
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contained  in  other  applications.  The  LAAs  recommended  for 
deletion  are  Built-up  Areas,  Slopes  >  10%,  Dense/Old  Forest,  and 
NO-GO  Gaps. 

It  is  also  recommended  that  a  capability  be  provided  to  edit 
the  various  LAAs  displays  in  such  a  way  that  feature  types  could 
be  suppressed,  displays  in  certain  areas  could  be  suppressed,  and 
displays  from  various  LAAs  could  be  combined  into  one  screen 
display.  This  capability  would  assist  the  user  in  developing 
Combined  Obstacle  Overlays  (COOs) . 

Finally,  it  is  recommended  that  the  suggestions  made  by  SMEs 
pertaining  to  specific  applications  be  evaluated  for  possible 
implementation . 


20 


REFERENCES 


Biirroughs,  P.  A.  (1990) .  Principles  of  geographic  information 
systems  for  land  resource  assessment.  Washington,  DC; 
Clarendon  Press. 

Command  and  General  Staff  College.  (1992) .  The  command  estimate 
process .  ST  100-9. 

Lockheed  Austin  Division.  (May  1992) .  Draft  software  user^s 

manual  for  the  AirLand  Battle  Management  Advanced  Technology 
Transition  Demonstration  Force  Level  Control  fALBM  ATP  FLCl 
advisor  system,  version  0.1.  Austin,  TX. 

Flanagan,  J.  P.  (In  preparation) .  Evaluation  of  ALBM  ATP 

prototype  version  1.2;  Review  of  the  Trend  Analysis  and 
Projection  Approach  Generation  Tool.  ARI  Research  Product. 
Alexandria,  VA;  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences. 

McKeown,  P.  E.  (In  preparation) .  Evaluation  of  ALBM  ATP 

prototype  version  1.2;  Knowledge  base  assessment  of  the 
Avenue  of  Approach  Generation  Tool.  ARI  Research  Product. 
Alexandria,  VA;  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences. 

Rappold,  V.  A.,  &  Flanagan,  J.  P.  (In  preparation).  Evaluation 

of  ALBM  ATP  prototype  version  1.2;  Human  factors  assessment 
of  the  Soldier  Machine  Interface.  ARI  Research  Product. 
Alexandria,  VA;  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences. 

Riedel,  S.  L. ,  McKeown,  P.  E.,  Flanagan,  J.  P. ,  &  Adelman,  L. 

(In  preparation) .  Evaluation  of  ALBM  ATP  prototype  version 
1.2;  Knowledge  base  assessment  of  the  Avenue  of  Approach 
Comparison  Tool.  ARI  Research  Product.  Alexandria,  VA; 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences. 


21 
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terrain  is  NO-GO  even  if  all  other  attributes  are  in  the  GO  category.  If  the  terrain  is  not  open  (e.g.  restrictive  tree 
diameter  and  spacing),  then  the  existence  of  roads/trails  may  have  the  effect  of  raising  the  mobility  of  the  terrain.  For 
example,  if  two  roads  exist  in  an  area,  then  restrictive  tree  diameter  and  spacing  would  not  reduce  the  mobility  to 
SLOW-GO  or  NO-GO. 
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APPENDIX  B 

SCREEN  IMAGES  OF  LOCATION  ANALYSIS  TOOLS 
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Figure  B-1 .  Background  Map 
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Fij'.ure  B-A.  Fross-C^iutit  rv  Mobil  itv. 


Map  Background  Operation 


B-6 


Figure  B-5.  Surface  Configuration  (Slopes). 


Fi^'ure  B-6.  L  i  ne-o  f -S  i  gh  t  ,  Not  Masked. 


B-9 


Figure  B-8.  Location  Analysis  (Tactical  Assembly  Area). 
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Figure  B-13.  Transportation. 
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Figure  B-14.  Terrain  Artributes  (Area). 
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i-17.  Surface  Materials 


B-19 


B-20 


B-21 


'or-vi 


APPENDIX  C 

QUESTIONNAIRES  AND  RELEASE  FORMS 
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Location  Analysis  Questionaire 
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11  Is  the  display  usable,  in  terms  ol  information  content  and 

Terrain  ,o,^at? 


WALK  THROUGH  QUESTIONNAIRE 
Your  Background 


Grade: _ 

Time  in  Grade;  _ 
Current  Position: 


Date; 

Branch: _ 

Time  in  Service;  ■ _ 

Duties  qf  Current  Position; _ 


Highest  level  of  civilian  education; _ 

Area  of  study: _ _ _ _ 

Please  indicate  yeax<s)  of  attendance  for  military  schods  completed: 

Officer  Basic  Course:  _ _ 

Officer  Advanced  Course; 

Combined  Amts  &  Service  Swff  School  (CAS  3 _ 

Command  and  CSeneral  Staff  Course  (CGSOC  or  equivalent); _ 

War  College:  _ _ — _ _ _ 

Other  relevant  military  education: 

Please  indicate  roost  significant  tactical  command  or  staff  positions  held: 

Echelon  Unit  Type  Position  Months  in  Position 


Which  of  the  following  best  describes  your  experience  and  level  of  skill  with  computers? 
Less  than  1  year,  relatively  unfamiliar _ _ _ 

1  •  3  years,  somewhat  familiar _ _ _ 

More  than  3  years,  quite  familiar  _ 

N^Tiich  of  the  following  best  describes  how  you  now  do  tactical  planning?: 

Manually 

Use  Maneuver  Control  System  (MCS)  ’ _ 

Use  other  aided  systems  (please  list) _  _  T 
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VOLUNTEER  AGREEMENT 


I, _ _  (print  name),  hereby  volunteer  to 

participate  in  a  study  to  evaluate  the  ALBM  ATD  prototype.  FLCA  1.2,  untter  the  co¬ 
direction  of  Sharon  Riedel  from  the  Army  Research  Institute  and 
MAJ  M.  C  Berwanger  from  the  Batde  Command  Battle  LaboratOTy,  Ft.  Leavenwonh,  KS. 

The  study  has  been  explained  to  me  and  is  described  on  the  preceeding  page,  which  1  have 
signed.  I  have  been  given  the  qrportunity  to  ask  questions  concerning  this  investigational 
study,  and  any  such  questions  have  been  answered  to  my  complete  satisfaction. 

I  understand  that  I  may  at  any  time  during  the  course  of  this  study  revoke  my  consent,  and 
withdraw  from  the  study  without  prejudice. 


signature  date 


witness  signature  date 
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APPENDIX  D 
SUBJECT  MATTER  EXPERTS' 


BACKGROUNDS 
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APPENDIX  E 

LOCATION  ANALYSIS  APPLICATIONS  RATINGS  AND  COMMENTS 
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Location  Analysis  Questionnaire  Results 


E-2 


Is  the  display  usable,  in  terms  of  information  content  || 

and  format?  What  should  be  done  to  H 

III  I  improve  the  display,  and  why?  |  Verbal  Comments 

ighly  Rather  Border  Somewhat  Decidedly  (written  answer)  | 

table  Usable  line  Unusable  UnussMe  H 


E-3 


_  Is  the  display  usable,  in  terms  ol  intormation  content 

Terrain  and  format?  What  should  be  done  to 

Overlay/  i  •  improve  the  display,  and  why?  Verbal  Comments 

Query  Highly  Rather  Border  Somewhat  Decidedly  (written  answer) 

Usable  Usable  tine  Unusable  Unusable 


low  road  capability. 
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Is  the  display  usable,  in  terms  ol  information  content 

Terrain  format?  What  should  be  done  to 

Overlay/  i  ■  ■  improve  the  display,  and  why?  Verbal  Comments 

Query  Highly  Rather  Border  Somewhat  Oeddedly  (written  answer) 

Usable  Usable  line  Unusable  Unusable 


<  O  UJ 


Is  the  display  usable,  in  terms  of  information  content 

Terratn  format?  What  should  be  done  to 

Overlay/  •  I  improve  the  display,  and  why?  Verbal  Comments 

Query  Highly  Rather  Border  Somewhat  Decidedly  (written  answer) 

Usable  UsaUe  line  Unusable  Unusable 


Is  to  show  several  proposed 
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GLOSSARY  OF  ACRONYMS  AND  ABBREVIATIONS 
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Appendix  F 


GLOSSARY  OF  ACRONYMS  AND  ABBREVIATIONS 


AA 

AACT 

CCM 

ALBM 

AMC 

ARI 

ATCCS 

ATD 

BCBL 

C&C 

COA 

CECOM 

DMA 

EM 

ETL 

ESC 

FITE 

FLC 

FM 

FSC 

LAA 

LOS 

MCOO 

MCS 

MET4 

OCOKA 

OPORD 

PEO-CCS 

SD 

SME 

TRADOC 


Avenue  of  Approach 

Avenue  of  Approach  Comparison  Tool 

Cross  Country  Mobility 

AirLand  Battle  Management 

Amny  Materiel  Command 

Army  Research  Institute 

Army  Tactical  Command  and  Control  System 
Advanced  Technology  Demonstration 
BAttle  Command  Battle  Laboratory 
Cover  and  Concealment 
Course  of  Action 

Communications  and  Electronics  Command 
Defense  Mapping  Agency 
Execution  Monitor 

Engineering  Topographic  Labortory 
Enemy  Situation  Capabilities 
Force  Interactive  Tactical  Evaluator 
Force  Level  Control 
Field  Manual 

Friendly  Situation  Capabilities 
Location  Analysis  Applications 
Line  of  Sight 

Modified  Combined  Obstacle  Overlay 
Maneuver  Control  System 

Mission,  Enemy,  Terrain,  Troops  and  Time  Available 
Applications 

Observation  and  Fire,  Cover  and  Concealment, 
Obstacles,  Key  Terrain,  Adequacy  of  Maneuver  Space 
Operations  Order 

Program  Executive  Office  for  Command  and  Control 
Systems 

Standard  Deviation 
Subject  Matter  Expert 
Training  and  Doctrine  Command 
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FIRST  TIME  DISTRIBUTION  LIST 
FOR 


EVALUATION  OF  AIRLAND  BATTLE  MANAGEMENT  ADVANCED  TECHNOLOGY 
DEMONSTRATION  PROTOTYPE  VERSION  1.2:  KNOWLEDGE  BASE  ASSESSMENT 
OF  THE  LOCATION  ANALYSIS  APPLICATIONS. 
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